The widespread adoption of IoT has driven the development of cyber-physical systems (CPS) in industrial environments, leveraging Industrial IoTs (IIoTs) to automate manufacturing processes and enhance productivity. The transition to autonomous systems introduces significant operational costs, particularly in terms of energy consumption. Accurate modeling and prediction of IIoT energy requirements are critical, but traditional physics- and engineering-based approaches often fall short in addressing these challenges comprehensively. In this paper, we propose a novel methodology for benchmarking and analyzing IIoT devices and applications to uncover insights into their power demands, energy consumption, and performance. To demonstrate this methodology, we develop a comprehensive framework and apply it to study an industrial CPS comprising an educational robotic arm, a conveyor belt, a smart camera, and a compute node. By creating micro-benchmarks and an end-to-end application within this framework, we create an extensive performance and power consumption dataset, which we use to train and analyze ML models for predicting energy usage from features of the application and the CPS system. The proposed methodology and framework provide valuable insights into the energy dynamics of industrial CPS, offering practical implications for researchers and practitioners aiming to enhance the efficiency and sustainability of IIoT-driven automation.

We present a digital-twin simulator for a pastillation process. The simulation framework produces realistic thermal image data of the process that is used to train computer vision-based soft sensors based on convolutional neural networks (CNNs); the soft sensors produce output signals for temperature and product flow rate that enable real-time monitoring and feedback control. Pastillation technologies are high-throughput devices that are used in a broad range of industries; these processes face operational challenges such as real-time identification of clog locations (faults) in the rotating shell and the automatic, real-time adjustment of conveyor belt speed and operating conditions to stabilize output. The proposed simulator is able to capture this behavior and generates realistic data that can be used to benchmark different algorithms for image processing and different control architectures. We present a case study to illustrate the capabilities; the study explores behavior over a range of equipment sizes, clog locations, and clog duration. A feedback controller (tuned using Bayesian optimization) is used to adjust the conveyor belt speed based on the CNN output signal to achieve the desired process outputs.

We present a novel reinforcement learning (RL) environment designed to both optimize industrial sorting systems and study agent behavior in evolving spaces. In simulating material flow within a sorting process our environment follows the idea of a digital twin, with operational parameters like belt speed and occupancy level. To reflect real-world challenges, we integrate common upgrades to industrial setups, like new sensors or advanced machinery. It thus includes two variants: a basic version focusing on discrete belt speed adjustments and an advanced version introducing multiple sorting modes and enhanced material composition observations. We detail the observation spaces, state update mechanisms, and reward functions for both environments. We further evaluate the efficiency of common RL algorithms like Proximal Policy Optimization (PPO), Deep-Q-Networks (DQN), and Advantage Actor Critic (A2C) in comparison to a classical rule-based agent (RBA). This framework not only aids in optimizing industrial processes but also provides a foundation for studying agent behavior and transferability in evolving environments, offering insights into model performance and practical implications for real-world RL applications.

Abstract-- Centralized control of a multi-agent system improves upon distributed control especially when multiple agents share a common task e.g., sorting different materials in a recycling facility. Traditionally, each agent in a sorting facility is tuned individually which leads to suboptimal performance if one agent is less efficient than the others. Centralized control overcomes this bottleneck by leveraging global system state information, but it can be computationally expensive. In this work, we propose a novel framework called Longitudinal Control Volumes (LCV) to model the flow of material in a recycling facility. We then employ a Kalman Filter that incorporates local measurements of materials into a global estimation of the material flow in the system. In this work, we present a novel modeling framework that divides the entire conveyor into smaller Longitudinal Centralized control approaches improve task performance Control Volumes (LCV) which enables a low-dimensional over distributed approaches by leveraging information about representation and prediction of material flows throughout the system state to coordinate agent actions.

Intelligent manufacturing is becoming increasingly important due to the growing demand for maximizing productivity and flexibility while minimizing waste and lead times. This work investigates automated secondary robotic food packaging solutions that transfer food products from the conveyor belt into containers. A major problem in these solutions is varying product supply which can cause drastic productivity drops. Conventional rule-based approaches, used to address this issue, are often inadequate, leading to violation of the industry's requirements. Reinforcement learning, on the other hand, has the potential of solving this problem by learning responsive and predictive policy, based on experience. However, it is challenging to utilize it in highly complex control schemes. In this paper, we propose a reinforcement learning framework, designed to optimize the conveyor belt speed while minimizing interference with the rest of the control system. When tested on real-world data, the framework exceeds the performance requirements (99.8% packed products) and maintains quality (100% filled boxes). Compared to the existing solution, our proposed framework improves productivity, has smoother control, and reduces computation time.

Game designers can use computer-aided game design methods to quantitatively compare player skill levels, different game variants, and learning rates, for the purpose of modeling how players will likely experience a game. We use Monte-Carlo simulation, hazard functions, and survival analysis to show how difficulty will quantitatively change throughout a game level as we vary skill, game parameters, and learning rates. We give a mathematical overview of survival analysis, present empirical data analyses of our player models for each game variant, and provide theoretical probability distributions for each game. This analysis shows the quantitative reasons why balancing a game for a wide range of player skill can be difficult; our player modeling provides tools for tuning this game balance. We also analyze the score distribution of over 175 million play sessions of a popular online Flappy Bird variant to demonstrate how learning effects can impact scores, implying that learning is crucial aspect of player modeling.